Search for: All records

High-quality environment lighting is essential for creating immersive mobile augmented reality (AR) experiences. However, achieving visually coherent estimation for mobile AR is challenging due to several key limitations in AR device sensing capabilities, including low camera FoV and limited pixel dynamic ranges. Recent advancements in generative AI, which can generate high-quality images from different types of prompts, including texts and images, present a potential solution for high-quality lighting estimation. Still, to effectively use generative image diffusion models, we must address two key limitations of content quality and slow inference. In this work, we design and implement a generative lighting estimation system called CleAR that can produce high-quality, diverse environment maps in the format of 360° HDR images. Specifically, we design a two-step generation pipeline guided by AR environment context data to ensure the output aligns with the physical environment's visual context and color appearance. To improve the estimation robustness under different lighting conditions, we design a real-time refinement component to adjust lighting estimation results on AR devices. To train and test our generative models, we curate a large-scale environment lighting estimation dataset with diverse lighting conditions. Through a combination of quantitative and qualitative evaluations, we show that CleAR outperforms state-of-the-art lighting estimation methods on both estimation accuracy, latency, and robustness, and is rated by 31 participants as producing better renderings for most virtual objects. For example, CleAR achieves 51% to 56% accuracy improvement on virtual object renderings across objects of three distinctive types of materials and reflective properties. CleAR produces lighting estimates of comparable or better quality in just 3.2 seconds---over 110X faster than state-of-the-art methods. Moreover, CleAR supports real-time refinement of lighting estimation results, ensuring robust and timely updates for AR applications. 

Abstract Agricultural researchers are increasingly encouraged to engage with stakeholders to improve the usefulness of their projects, but iterative research on the design and assessment of stakeholder engagement is scarce. The USDA Long‐Term Agroecosystem Research (LTAR) Network recognizes the importance of effective engagement in increasing the utility of information and technologies for future agriculture. Diverse stakeholders and researchers at the Kellogg Biological Station (KBS) LTAR site co‐designed the KBS LTAR Aspirational Cropping System Experiment, a process that provides a testing ground and interdisciplinary collaborations to develop theory‐driven assessment protocols for continuous stakeholder engagement. Informed by prior work, we designed an assessment protocol that aims to measure participant preferences, experiences, and perceived benefits at various stages of this long‐term project. Two online surveys were conducted in 2021 and 2022 among participants of LTAR engagement events at KBS, using a pre‐post design, resulting in 125 total responses. Survey respondents had positive perceptions of the collaboratively designed research experiment. They had a strong expectation that the research would generate conservation and environmental advances while also informing policy and programs. Respondents also indicated a desire to network with other stakeholders. The research team noted the significant role of a long‐term stakeholder engagement specialist in inviting participants from diverse backgrounds and creating an open and engaging experience. Overall, results highlight an interdisciplinary path of intentional and iterative engagement and evaluation to build a program that is adaptive and responsive to stakeholder needs.